The present invention relates to a stereoscopic image display apparatus (stereoscopic display apparatus) and, more particularly, to an apparatus which can be suitably used to stereoscopically display image information on the display device (display) of a TV set, a video recorder, a computer monitor, a game machine, or the like and allows an observer to stereoscopically observe the image information from a predetermined observation region.
The conventional stereoscopic image observation methods which have been proposed include a method of observing parallax images in different polarization states by using polarization glasses, a method of guiding predetermined parallax images, of a plurality of parallax images (viewpoint images), to the eyes of an observer by using a lenticular lens, and the like.
In a stereoscopic display scheme using polarization glasses, a right-eye parallax image and a left-eye parallax image are displayed in different polarization states, and the observer uses the polarization glasses to separate the right and left parallax images so as to visually recognize them as a stereoscopic image. For example, a liquid crystal shutter is placed on the display side to set right and left parallax images in different parallax states, and the parallax states are switched in synchronism with each field signal of the image displayed on the display. With this operation, an observer with polarization glasses can separately and alternately observe right and left images with a corresponding eye time-divisionally so as to realize a stereoscopic view.
A method by placing a liquid crystal shutter on the polarization glasses side has also been proposed. In this method, the shutter is operated in synchronism with the image displayed on a monitor to allow an observer to visually recognize right and left parallax images properly so as to recognize a stereoscopic image.
In a stereoscopic display scheme using a lenticular lens, the lenticular lens is placed on the front side of an image display surface to spatially separate an image into images for the right and left eyes of an observer so as to realize a stereoscopic vision.
FIG. 46 is a view for explaining a stereoscopic display apparatus of the conventional lenticular lens scheme. Referring to FIG. 46, reference numeral 151 denotes a liquid crystal display, in which a display pixel portion 153 of the liquid crystal is formed between glass substrates 152 and 154. A lenticular lens 155 constituted by many cylindrical lenses, each having a semicircular cross-section and extending in a direction perpendicular to the drawing surface as shown in FIG. 46, is formed on the observer side of the liquid crystal display 151. The display pixel portion 153 of the liquid crystal is positioned on the focal plane of the lenticular lens 155.
A striped image is displayed in the display pixel portion 153. A striped image is synthesized from a plurality of parallax images (images with parallaxes) from a plurality of viewpoints. To synthesize a striped image, at least two parallax images are required. Let Rs be a parallax image corresponding to the right eye, and Ls be a parallax image corresponding to the left eye. Each parallax image is divided into elongated stripe pixels (to be referred to as stripe pixels hereinafter) Ri and Li (i=1, 2, 3, . . . ). The stripe pixels obtained from the respective parallax images are alternately arranged, i.e., arranged in the following sequence: the stripe pixels R1, L2, R3, L4, . . . (or L1, R2, L3, R4, . . . ), into one image, thus forming a striped image. A xe2x80x9cthree-dimensional imagexe2x80x9d in this specification indicates this striped image. The formation of a striped image will be referred to as stripe synthesis hereinafter.
If three parallax images A, B, and C are prepared, a striped image is obtained by alternately arranging stripe pixels in the following sequence: stripe pixels A1, B2, C3, A4, B5, C6, . . . , B1, C2, A3, B4, C5, A6, . . . , or C1, A2, B3, C4, A5, B6, . . .
As shown in FIG. 46, the display pixel portion 153 is obtained by alternately arranging right-eye stripe pixels (black portions) and left-eye stripe pixels (white portions), each having the form of a stripe and extending in the direction perpendicular to the drawing surface, in correspondence with the pitch of the lenticular lens. The lenticular lens 155 optically separates light beams from these stripe pixels into regions in which right and left eyes ER and EL of the observer are present. With this operation, a stereoscopic vision is realized.
FIG. 46 shows a spatial region where right-eye and left-eye images on the two end portions and central portion of the liquid crystal display 151 can be observed. Referring to FIG. 46, a stereoscopic vision region 156 indicated by the thick solid lines is a common region where right and left images are separately observed with the eyes (the distance between the centers of the eyes is represented by e) of the observer throughout the entire screen. Note that another stereoscopic vision region (not shown) where a stereoscopic vision can be obtained is also present near the stereoscopic vision region 156.
In addition to the above lenticular lens scheme, the parallax barrier scheme is available as a stereoscopic display scheme. This parallax barrier scheme is described below.
A stereoscopic image display scheme using the parallax barrier scheme is disclosed by. S. H. Kaplan (xe2x80x9cTheory of Parallax Barriers.xe2x80x9d, J. SMPTE, Vol. 59, No. 7, pp. 11-21, 1952). In this scheme as well, a striped image synthesized from right and left parallax images like those described above is displayed, and an observer separately observes the parallax images corresponding to the right and left eyes with the corresponding eyes through a slit pattern (called a parallax barrier) having predetermined opening portions placed at a predetermined distance from this striped image, thereby realizing a stereoscopic vision.
Japanese Patent Laid-Open Nos. 3-119889 and 5-122733 disclose stereoscopic display apparatuses in which a parallax barrier is electronically formed by a transmission type liquid crystal display device and the like, and the shape, position, and the like of the parallax barrier are electronically controlled and changed.
FIG. 47 shows the schematic arrangement of the main part of the stereoscopic image display apparatus disclosed in Japanese Patent Laid-Open No. 3-119889. In this apparatus, an electronic parallax barrier 103 constituted by a transmission liquid crystal display device is placed on an image display surface 101 through a spacer 102 having a thickness d. A vertically striped image constituted by parallax images obtained from two or more directions is displayed on the image display surface 101. A parallax barrier is formed at an arbitrary position on the barrier surface of the electronic parallax barrier 103 upon designation of X and Y addresses by a control means such as a microcomputer 104, thereby allowing a stereoscopic view based on the principle of the above parallax barrier scheme.
FIG. 48 is a schematic view showing the arrangement of the main part of the stereoscopic image display apparatus disclosed in Japanese Patent Laid-Open No. 3-119889, which is constituted by a liquid crystal display and an electronic barrier. In this stereoscopic image display apparatus, two liquid crystal layers 115 and 125 are respectively sandwiched between two polarizing plates 111 and 118 and between two polarizing plates 121 and 128. The liquid crystal layer 115 and the liquid crystal layer 125 respectively serve as an image display means and an electronic barrier formation means.
An apparatus capable of performing mixed display of three- and two-dimensional images within a single display surface is disclosed in Japanese Patent Laid-Open No. 5-122733. In this apparatus, as shown in FIG. 49, a striped barrier pattern can be generated only in any partial area of an electronic parallax barrier 103 constituted by a transmission type liquid crystal display device.
In a stereoscopic display apparatus capable of realizing a stereoscopic vision by forming and displaying a striped image as in the above lenticular lens scheme and the parallax barrier scheme, the width of the region which allows a stereoscopic vision is small, and the maximum area in which an observer can obtain a stereoscopic vision is as small as xc2xd the distance between the centers of the eyes, which is about 65 mm. For this reason, the observer must fix the position of the head portion, and has difficulty in stably obtaining a stereoscopic vision.
To solve this problem, a scheme for widening such a stereoscopic vision region is disclosed in Japanese Patent Laid-Open No. 2-44995. According to this scheme, to widen this stereoscopic vision region, the positions of the eyes of an observer are detected, and the relative positions of the lenticular lens and the display device in the right-and-left direction are moved/controlled.
In another scheme disclosed in Japanese Patent Laid-Open No. 2-50145, the positions of the eyes of an observer are detected, and the lateral positions of stripe pixels constituting right-eye and left-eye parallax images corresponding to the lenticular lens are interchanged in accordance with the resultant signal, thereby obtaining a wide stereoscopic vision region.
A means for detecting the positions of the eyes of an observer and obtaining a wide stereoscopic vision region will be briefly described below. For example, an image processing method for capturing the observer with a camera and extracting a contour from the captured image or detecting the observer by pattern matching has been proposed as a means for detecting the positions of the eyes of an observer.
FIG. 50 is a view for explaining the principle of a technique of making a lenticular lens follow up the movement of an observer in the right-and-left direction in the lenticular lens scheme. Referring to FIG. 50, reference numeral 160 denotes an observer before the movement; 160a, the position of the observer after he/she has moved from a predetermined position by a distance a in the right-and-left direction; 161, one of the cylindrical lenses constituting the lenticular lens; and 161a, the cylindrical lens after it has moved following up the movement of the observer. Reference symbol b denotes the movement amount of the cylindrical lens (=lenticular lens) in this case. Reference numeral 162 denotes a display pixel portion for displaying right stripe pixels (black portions) and left stripe pixels (white portions). Reference symbol f denotes the focal length of the cylindrical lens; and S, an observation distance.
Assume that a relation of S greater than  greater than f is satisfied. In this case, in general, when the lenticular lens is moved by the amount b upon movement of the observer by the distance a in the right-and-left direction,
b=fxc2x7a/sxe2x80x83xe2x80x83(1)
the stereoscopic vision region also moves by the distance a in the right-and-left direction.
In this case, the lens is moved. However, the same effect as that described above can be obtained even if the display pixel portion 162 is moved with respect to the lenticular lens.
The above description explains the principle of the technique of making the stereoscopic vision region follow the movement of the observer in the right-and-left direction with respect to the stereoscopic display. In some case, however, the observer moves in the direction of depth as well as the right-and-left direction with respect to the stereoscopic display, and may move out of the stereoscopic vision region in the direction of depth.
As a technique of solving this problem, a scheme of projecting/displaying a three-dimensional image on a lenticular lens is disclosed in Japanese Patent Laid-Open No. 4-122922. According to this scheme, the position of an observer in the direction of depth is also detected to realize stereoscopic vision follow-up in the direction of depth.
As means for detecting the position of an observer in the right-and-left direction, the following methods are disclosed in Japanese Patent Laid-Open No. 2-50145:
(a1) a method of mainly irradiating infrared light and detecting reflected light;
(a2) a method of irradiating infrared light onto an observer, and receiving the light reflected by an observer with a linear CCD image sensor, thereby detecting the position of the observer;
(a3) a method of irradiating infrared light from the rear side of the observer, and detecting his/her position from the light amount distribution of a light-receiving device placed on the front surface side; and
(a4) a method of performing contour extraction processing for an image of an observer using a TV camera, and detecting the positions of the eyes by an image recognition technique.
As means for detecting the position of an observer in the direction of depth, the following means are disclosed in Japanese Patent Laid-Open No. 4-122922:
(a5) a method of using two distance detectors using infrared light; and
(a6) a method of performing image processing by using two cameras.
The following problems, however, are posed in the above conventional stereoscopic display apparatuses.
(b1) According to the scheme of capturing an observer with a camera or the like and detecting the position of the observer by image processing, high-speed detection cannot be performed, and the follow-up speed is low. That is, satisfactory follow-up characteristics cannot be ensured. In addition, since a high-performance arithmetic function is required for the apparatus based on this scheme, the overall apparatus tends to be complicated.
(b2) In the scheme for obtaining the contour of the head portion of an observer in one direction, and detecting the viewpoint position by image processing, since the central position of the contour of the head portion is regarded as the middle point between the eyes, the central position (viewpoint position) of the eyes cannot be correctly detected depending on the hair style of the observer, the build of the observer, or the direction of the face of the observer.
(b3) To detect the contour of the head portion of an observer or the positions of the eyes, a camera or a light-receiving means must capture the head portion or eyes of the observer. If, therefore, a viewpoint position follow-up mechanism designed to perform a follow-up operation only in the right-and-left direction is to be used, the position of the observer must be checked or adjusted such that the observer is positioned in the up-and-down direction within the visual field of the camera or the light-receiving means in the initial state. There is, however, no conventional means for easily performing correct adjustment. In addition, there is no means for checking whether the observer has moved out of the visual field while is observing an image on the display.
(b4) In the scheme of capturing an observer with a camera or the like and detecting the observation position by image processing, a recognition error may be caused by an object in the background of the observer. It is therefore difficult to detect the correct position of the observer.
(b5) To capture the movement of the contour of the head portion of an observer, the visual field of the camera must be larger than the contour of the head portion. To cope with the large movement of the head portion, the visual field must be widened accordingly. For this reason, an image sensing device with a large number of pixels is required to increase the precision of position detection. It therefore takes much time to perform image processing.
(b6) In the method of extracting the contour of the head portion of an observer, even slight rotation of the head portion of the observer may cause a detection error, resulting in failure to follow the stereoscopic vision region.
It is, therefore, an object of the present invention to provide a stereoscopic image display apparatus, in which the arrangement of a viewpoint position detection sensor for emitting visible light or infrared light to an observer and detecting reflected light from the observer, the arrangement of a follow-up control means for performing follow-up control on a portion of the apparatus by using the signal obtained by the viewpoint position detection sensor, and the like are properly set to accurately detect the viewpoint position of the observer with a simple arrangement and easily cause a stereoscopic view region to follow up at a high speed.
In order to achieve the above object, the present invention has the following arrangements. There is provided a stereoscopic image display apparatus which allows an observer to observe a stereoscopic image by guiding light beams based on a plurality of parallax images displayed on an image display unit to the observer through an optical system, comprising:
a viewpoint position detection sensor for detecting a viewpoint position of the observer; and
follow-up control means for performing follow-up control on a stereoscopic view observation region for image information displayed on the image display unit, on the basis of viewpoint position information obtained by the viewpoint position detection sensor,
the viewpoint position detection sensor including
light-emitting means for emitting a light beam to the observer,
narrow-area detection means for detecting a viewpoint position in a narrow area, and
wide-area detection means for detecting a viewpoint position in a wide area.
According to the stereoscopic image display apparatus, there can be provided a stereoscopic image display apparatus, in which the arrangement of a viewpoint position detection sensor for emitting visible light or infrared light to an observer and detecting reflected light from the observer, the arrangement of a follow-up control means for performing controls to adjust the projection of the stereoscopic view observation region of an image displayed on the image display unit, on the basis of the viewpoint position information obtained by the viewpoint position detection sensor, and the like are properly set to accurately detect the viewpoint position of the observer with a simple arrangement and easily cause a stereoscopic view region to follow up at a high speed.
There is also provided a stereoscopic image display apparatus which allows an observer to observe a stereoscopic image by guiding light beams based on a plurality of parallax images displayed on an image display unit to the observer through an optical system, comprises:
a viewpoint position detection sensor for detecting the viewpoint position of the observer; and
follow-up control means for performing follow-up control on a stereoscopic view observation region for image information displayed on the image display unit, on the basis of viewpoint position information obtained by the viewpoint position detection sensor,
the viewpoint position detection sensor including
light-emitting means for emitting a light beam to the observer,
light-receiving means for receiving a light beam from the observer; and
position adjustment means for adjusting at least one of a light-emitting area of the light-emitting means and a light-receiving area of the light-receiving means to predetermined positions.
There is also provided a stereoscopic image display apparatus which allows an observer to observe a stereoscopic image by guiding light beams based on a plurality of parallax images displayed on an image display unit to the observer through an optical system, comprises:
a viewpoint position detection sensor for detecting the viewpoint position of the observer; and
follow-up control means for performing follow-up control on a stereoscopic view observation region for image information displayed on the image display unit, on the basis of viewpoint position information obtained by the viewpoint position detection sensor,
the viewpoint position detection sensor including
light-emitting means for emitting a light beam to the observer,
narrow-area detection means for detecting a viewpoint position in a narrow area on an observer side, and
wide-area detection means for detecting a viewpoint position in a wide area,
wherein the viewpoint position is finally determined in accordance with a signal from the narrow-area detection means.
There is also provided a stereoscopic image display apparatus which allows an observer to observe a stereoscopic image by guiding light beams based on a plurality of parallax images displayed on an image display unit to the observer through an optical system, comprises:
a viewpoint position detection sensor for detecting the viewpoint position of the observer; and
follow-up control means for performing follow-up control on a stereoscopic view observation region for image information displayed on the image display unit, on the basis of viewpoint position information obtained by the viewpoint position detection sensor,
wherein the viewpoint position detection sensor forms images from a plurality of directions of the observer on a line sensor having a plurality of light-receiving elements arranged in one direction, and position information of the observer in one direction and information of a distance from the viewpoint position detection sensor to the observer are obtained.
There is also provided a stereoscopic image display apparatus which allows an observer to observe a stereoscope image by guiding light beams based on a plurality of parallax images displayed on an image display unit to observer through an optical system, comprising:
viewpoint position detecting means for detecting a viewpoint position of the observer; and
a controller for performing follow-up control on a stereoscopic view observation region for image information displayed on the image display unit, on the basis of viewpoint position information obtained by the viewpoint position detecting means,
wherein the viewpoint position detecting means having a function of detecting the viewpoint in a narrow area and a function of detecting the viewpoint position in a wide area.
According to a preferred aspect of the present invention, the follow-up control means controls the viewpoint position detection sensor to follow up a viewpoint position of the observer.
According to another preferred aspect of the present invention, the light-emitting means further comprises means for limiting a light-emitting area with respect to the observation side.
According to still another preferred aspect of the present invention, the light-emitting means emits a light beam modulated with a predetermined light-emitting frequency, and the narrow-area detection means or the wide-area detection means further comprises synchronization means for detecting signals on the basis of the frequency modulation.
According to still another preferred aspect of the present invention, a parallax image is displayed on the image display unit at a predetermined display frequency, the display frequency being set so as not to be an integer multiple of the light-emitting frequency.
According to still another preferred aspect of the present invention, the light-emitting means controls the intensity of a light beam to be emitted such that a signal obtained by the narrow-area detection means or the wide-area detection means on the basis of a reflected light beam from the observer is not less than a predetermined value.
According to still another preferred aspect of the present invention, each of the narrow-area detection means and the wide-area detection means comprises a pair of light-receiving elements, and adjusts a distance between the pair of light-receiving elements to detect a viewpoint position in a narrow or wide area.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor detects a viewpoint position in the narrow or wide area by using at least one of a focal length of a light-emitting lens for irradiating a light beam to the observer, a focal length of a light-receiving means for receiving a light beam from the observer, slit means for limiting a light-receiving area for the light beam from the observer side, and directivity characteristics of a light source for emitting a light beam to the observer.
According to still another preferred aspect of the present invention, a stereoscopic view of image information displayed on the image display unit is realized by using a lenticular lens or a parallax barrier, and the lenticular lens or parallax barrier is driven/controlled by using a sum total of different signals based on outputs from the pair of light-receiving elements of the narrow-area detection means and difference signals based on outputs from the pair of light-receiving elements of the wide-area detection means.
According to still another preferred aspect of the present invention, each of the light-emitting area and the light-receiving area includes a display member for adjusting a position of the viewpoint position detection sensor to a reference position which is a predetermined position of the observer, and an observation window for observing the display member.
According to still another preferred aspect of the present invention, the light-emitting means comprises a light-emitting element, the light-emitting means also serving as the display member.
According to still another preferred aspect of the present invention, an output from the wide-area detection means becomes not more than a predetermined threshold when the observer is present within a predetermined parallax angle.
According to still another preferred aspect of the present invention, the light-emitting means comprises one or two point light sources.
According to still another preferred aspect of the present invention, the narrow-area detection means comprises a two-dimensional image sensing element for converting a face image of the observer into an electrical signal.
According to still another preferred aspect of the present invention, the light-emitting means emits visible light or infrared light modulated with a predetermined light-emitting frequency, and the wide-area detection means and the narrow-area detection means further comprise synchronization means for detecting a signal on the basis of the light-emitting frequency.
According to still another preferred aspect of the present invention, the two-dimensional image sensing element outputs information of a maximum luminance position or eyeball position in a predetermined region of the observer.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor follows movement of the viewpoint position of the observer.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor comprises a plurality of pairs each constituted by an optical system for focusing a light beam from the observer, and a line sensor for photoelectrically converting an image of the observer which is formed by the optical system.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor comprises an optical system for focusing a light beam from the observer, and a plurality of line sensors for photoelectrically converting an image of the observer which is formed by the optical system.
According to still another preferred aspect of the present invention, optical axes of the plurality of optical systems are substantially parallel to each other.
According to still another preferred aspect of the present invention, the observer is positioned near an intersection of optical axes of the plurality of optical systems which define a vergency angle.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor obtains a size of an image formed on a surface of the line sensor in accordance with a signal from the line sensor, and checking by using the size information whether the image is an image of the observer.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor obtains movement information of an image formed on a surface of the line sensor in accordance with a signal from the line sensor, and obtains the depth amount of the image on the observer side in an interval in which movement is detected.
According to still another preferred aspect of the present invention, the viewpoint position detection sensor obtains a correlation value from an image of the observer which is formed on the line sensor by template matching, and obtains depth information of the observer by using the correlation value.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.